Development of high integrity and mission-critical software systems requires the rigorous and efficient testing techniques. Even more challenging task is to ensure high quality requirements within embedded computing systems where different resources constraints exist. White-box testing techniques, such as code coverage analysis is, partially meet these challenges but its application is very often associated with additional load on target systems and thus achievable in far from real environments. Either it is hard to get the required thoroughness of analysis without stressing target system too much.
A lot of test coverage analysis tools already exist. Most of the already known tools in this field addressing basically the following problems:
1. Application of test coverage analysis in testing of embedded software in real environment.
2. Support of hardware with strict computing resources constraints.
3. Providing more thorough coverage metric than statement and decision coverage with reasonable efforts.
4. Achieving the higher trustworthiness of coverage data with checking of program flow correctness.
5. Semi-automatic execution of test coverage measurement.
The paper Cornett, S., Code Coverage Analysis, [5 Jun. 2008] gives a complete description of code coverage analysis or test coverage analysis. BullseyeCoverage tool which is proposed requires the file system functionality on target and thus cannot be used for simpler applications.
From the publication Testwell CTC++, Test Coverage Analyzer for C/C++, [5 Jun. 2008].
A system is known which supports communication interfaces for transferring coverage data from target to host but still requires operating system functionality on target.
The Manual for CoverageMeter, Code Coverage Measurement for C/C++, CoverageMeter Software Factory, [5 Jun. 2008] provides the state-of-the-art code coverage technique (source code analysis, instrumentation, generation of application), but still requires operating system functionality on target.
The tool suite Cantata++v.5.2.1, Technical brief, IPL, [5 Jun. 2008], along with other testing features is capable to perform integrated coverage analysis. It declares the possible communication of results data from target to host platform by means of serial, LAN, USB, JTAG. This tool does not require file I/O on the target platform but exact significant memory resources from target.
The publication Majzik I., Pataricza A. Control Flow Checking in Multitasking Systems//Periodica Polytechnica Ser. Electrical Engineering, Vol. 39, No. 1, pp 27-36, 1995 shows a new watchdog-processor method SETS (Signature Encoded Instruction Stream) and the part of the method responsible for instrumentation is based on a Assigned Signature Scheme.
None of the listed coverage analysis tools fully addresses the problems mentioned above. First, all known tools do not provide the coverage metric reporting on decisions interaction which contains potential bugs not discovered by other coverage metrics. Program flow correctness check is not provided by these tools as well. None of the tools provides instrumentation granularity up to a program basic block.